Registration of scanning data acquired from different patient positions

ABSTRACT

A system and method are provided for generating a three-dimensional visualization image of an object such as an organ using volume visualization techniques and exploring the image using a guided navigation system that allows the operator to travel along a flight path and to adjust the view to a particular portion of the image of interest in order, for example, to identify polyps, cysts or other abnormal features in the visualized organ; where imaging data sets are acquired using conventional two-dimensional scans of the object in both a supine and prone orientation, and correspondence between the respective data sets is determined to permit jumping from one visualization orientation to the other while remaining at the same virtual location within the organ.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit under 35 U.S.C. § 365(c) of PCTInternational Application Designating the United States, Serial No.PCT/US02/37387, entitled “REGISTRATION OF SCANNING DATA ACQUIRED FROMDIFFERENT PATIENT POSITIONS”, and filed Nov. 21, 2002, which isincorporated herein by reference in its entirety; and which, in turn,claims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional PatentApplication Ser. No. 60/331,712, entitled “New Features For VirtualColonoscopy”, filed Nov. 21, 2001, which is incorporated herein byreference in its entirety.

BACKGROUND OF THE INVENTION

The present disclosure relates to a system and method for performing avolume based three-dimensional virtual examination, and moreparticularly relates to a method providing enhanced visualization andnavigation properties.

Two-dimensional (“2D”) visualization of human organs using medicalimaging devices has been widely used for patient diagnosis. Currentlyavailable medical imaging devices include computed tomography (“CT”) andmagnetic resonance imaging (“MRI”), for example. Three-dimensional(“3D”) images can be formed by stacking and interpolating betweentwo-dimensional pictures produced from the scanning machines. Imaging anorgan and visualizing its volume in three-dimensional space would bebeneficial due to the lack of physical intrusion and the ease of datamanipulation. However, the exploration of the three-dimensional volumeimage must be properly performed in order to fully exploit theadvantages of virtually viewing an organ from the inside.

When viewing the 3D volume virtual image of an environment, a functionalmodel must be used to explore the virtual space. One possible model is avirtual camera, which can be used as a point of reference for the viewerto explore the virtual space. Camera control in the context ofnavigation within a general 3D virtual environment has been previouslystudied. There are two conventional types of camera control offered fornavigation of virtual space. The first gives the operator full controlof the camera, which allows the operator to manipulate the camera indifferent positions and orientations to achieve the view, desired. Theoperator will in effect pilot the camera. This allows the operator toexplore a particular section of interest while ignoring other sections.However, complete control of a camera in a large domain would be tediousand tiring, and an operator might not view all the important featuresbetween the start and finishing point of the exploration.

The second technique of camera control is a planned navigational method,which assigns the camera a predetermined path to take and which cannotbe changed by the operator. This is akin to having an engaged“autopilot”. This allows the operator to concentrate on the virtualspace being viewed, and not have to worry about steering into walls ofthe environment being examined. However, this second technique does notgive the viewer the flexibility to alter the course or investigate aninteresting area viewed along the flight path.

It would be desirable to use a combination of the two navigationtechniques described above to realize the advantages of both techniqueswhile minimizing their respective drawbacks. It would be desirable toapply a flexible navigation technique to the examination of human oranimal organs that are represented in virtual 3D space in order toperform a non-intrusive painless and thorough examination. The desirednavigational technique would further allow for a complete examination ofa virtual organ in 3D space by an operator, allowing flexibility whileensuring a smooth path and complete examination through and around theorgan. It would be additionally desirable to be able to display theexploration of the organ in a real time setting by using a techniquethat minimizes the computations necessary for viewing the organ. Thedesired technique should also be equally applicable to exploring anyvirtual object.

In practical application, radiological scanning equipment, for example,is used to acquire respective scan data from a patient from both asupine (i.e., face up) and prone (i.e., face down) position. Other thanbeing gathered from the same patient, the image data corresponding tothe two orientations are completely independent. As a consequence, whenan interesting feature is located in one orientation or view, there iscurrently no way to automatically “jump” to that same feature in theother orientation or view.

SUMMARY OF THE INVENTION

A preferred embodiment of the present disclosure generates athree-dimensional visualization image of an object such as a human organusing volume visualization techniques and explores the virtual imageusing a guided navigation system, which allows the operator to travelalong a predefined flight path and to adjust both the position andviewing angle to a particular portion of interest in the image away fromthe predefined path in order to identify polyps, cysts or other abnormalfeatures in the organ.

In accordance with a navigation method for virtual examination, afly-path through a virtual organ, such as a colon lumen, is generated.From the volume element representation of the colon lumen, volumeshrinking from the wall of the virtual colon lumen is used to generate acompressed colon lumen data set. From the compressed colon lumen dataset, a minimum distance path is generated between endpoints of thevirtual colon lumen. Control points are then extracted along the minimumdistance path along the length of the virtual colon lumen. The controlpoints are then centered within the virtual colon lumen. Finally, a lineis interpolated between the centered control points to define the finalnavigation fly-path.

In the above method for generating a fly-path, the step of volumeshrinking can include the steps of representing the colon lumen as aplural stack of image data; applying a discrete wavelet transformationto the image data to generate a plurality of sub-data sets withcomponents at a plurality of frequencies, and then selecting the lowestfrequency components of the sub-data sets.

Another method for generating a fly-path through a virtual colon lumenduring virtual colonoscopy includes the step of partitioning the virtualcolon lumen into a number of segments. A point is selected within eachsegment and the points are centered with respect to a wall of thevirtual colon lumen. The centered control points are then connected toestablish the fly path.

A method for performing examination of a virtual colon lumen includes avolume rendering operation. For each viewpoint within the colon lumen,rays are cast from the viewpoint through each image pixel. The shortestdistance from the viewpoint to a wall of the colon lumen is determinedfor each ray. If the distance exceeds a predetermined sampling interval,the processing effects a jump along the ray by the distance and assignsa value based on an open space transfer function to the points along theray over the jumped distance. If the distance does not exceed thesampling interval, then the current points are sampled and displayableproperties are determined according to a transfer function.

In another method in accordance with the present virtual imagingdisclosure, a method of performing virtual colonoscopy includes the stepof acquiring an image data set of a region, including the colon, andconverting the image data to volume units. Those volume unitsrepresenting a wall of the colon lumen are identified and a fly-path fornavigating through the colon lumen is established. At least one transferfunction is then used to map color and opacity coefficients to the wallof the colon lumen. The colon can then be displayed along the fly-pathin accordance with the assigned transfer functions.

In a method of virtual colonoscopy, the step of generating a fly-pathcan include using volume shrinking from the wall of the virtual colonlumen to generate a reduced data set. From the reduced data set, aminimum distance path between endpoints of the virtual colon lumen isgenerated. Control points along a length of the virtual colon lumen canthen be assigned along the minimum distance path. The control pointswithin the virtual colon lumen are then centered and a line connectingthe centered control points is interpolated to complete the navigationalfly-path.

Alternatively, a fly-path can be generated by partitioning the virtualcolon lumen into a plurality of segments, selecting a point within eachsegment, centering each point with respect to the wall of the virtualcolon lumen, and connecting the centered points to establish thefly-path.

A system for three dimensional imaging, navigation and examination of aregion in accordance with the present disclosure includes an imagingscanner, such as an MRI or CT scanner, for acquiring image data. Aprocessor converts the image data into a plurality of volume elementsforming a volume element data set. The processor also performs thefurther steps of identifying those volume units representing a wall ofthe colon lumen, establishing a fly-path for navigating through saidcolon lumen, and applying at least one transfer function to map colorand opacities to the wall of the colon lumen. A display unit isoperatively coupled to the processor for displaying a representation ofthe region in accordance with the fly-path and the at least one transferfunction.

An alternate computer-based system for virtual examination, formed inaccordance with an embodiment of the present disclosure, is based on abus structure architecture. A scanner interface board is coupled to thebus structure and provides data from an imaging scanner to the bus. Mainmemory is provided which is also coupled to the bus. A volume renderingboard having locally resident volume rendering memory receives at leasta portion of the data from the imaging scanner and stores this data inthe volume rendering memory during a volume rendering operation. Agraphics board is coupled to the bus structure and to a display devicefor displaying images from the system. A processor is operativelycoupled to the bus structure and is responsive to the data from theimaging scanner. The processor converts the data from the imagingscanner into a volume element representation, stores the volume elementrepresentation in main memory, partitions the volume elementrepresentation into image slices, and transfers the volume elementpartitions to the volume rendering board.

One aspect of the present disclosure relates to a method for registeringa prone-view with a supine-view three-dimensional volume representationof at least one object. According to this aspect, for the prone-viewrepresentation, a first set of features along a prone-view centerline islocated, and for the supine-view representation, a second set offeatures along a supine-view centerline is located. Next, at least onefeature in the first set is matched with corresponding features in thesecond set; alternatively, at least one feature in the second set ismatched with corresponding features in the first set.

Another aspect of the present disclosure relates to a method forperforming a three-dimensional internal examination of at least oneorgan. According to this aspect, the organ is scanned with aradiological device to produce first and second scan data setsrepresentative of the organ in a prone and supine view, respectively. Aprone-view three-dimensional volume representation of the organcomprising volume elements is created from the prone-view scan data anda supine-view three-dimensional volume representation of the organcomprising volume elements is created from the supine-view scan data.Next, a prone-view centerline and a supine-view centerline within therespective three-dimensional representation are generated, and thevolume elements responsive to the respective centerline and operator'sinput during a guided navigation along one of the supine-view orprone-view three-dimensional representations is displayed in real time.Respective features are identified along each centerline and matched tocreate pairs of corresponding features. An operator can jump from acurrent location in one three-dimensional representation to a targetlocation in the other based on the relative location from respectivecorresponding features.

BRIEF DESCRIPTION OF THE DRAWINGS

Further objects, features and advantages of the present disclosure willbecome apparent from the following detailed description taken inconjunction with the accompanying figures showing preferred embodimentsof the disclosure, in which:

FIG. 1 shows a flow chart of the steps for performing a virtualexamination of an object, specifically a colon, in accordance with thedisclosure;

FIG. 2 shows an illustration of a “submarine” camera model whichperforms guided navigation in the virtual organ;

FIG. 3 shows a diagram illustrating a two dimensional cross-section of avolumetric colon which contains the remaining flight path;

FIG. 4 shows a diagram of a system used to perform a virtual examinationof a human organ in accordance with the disclosure;

FIG. 5 shows a block diagram of a system embodiment based on a personalcomputer bus architecture;

FIG. 6 illustrates a centerline profile with accompanying vectorsshowing, in accordance with an embodiment of the present disclosure, achange in direction around a voxel;

FIGS. 7A and 7B illustrate, respectively, supine and prone centerlineplots depicting a “z” profile of the centerline overlaid with a plot ofthe change in direction of the “z” profile as calculated according to anembodiment of the present disclosure;

FIG. 8A depicts a voxel z profile for both a supine view and a proneview according to an embodiment of the present disclosure;

FIG. 8B shows the profiles of FIG. 8A aligned at their hepatic flexuresby shifting one view in accordance with an embodiment of the presentdisclosure;

FIG. 9 depicts a supine view (similar to that of FIG. 7A) and a proneview (similar to FIG. 7B) each having two disjoint segments; and

FIG. 10 depicts a flowchart that summarizes a registration methodaccording to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

While the methods and systems described herein may be applied to anyobject to be examined, the preferred embodiment to be described is theexamination of an organ in the human body, specifically the colon. Thecolon is long and twisted, which makes it especially suited for avirtual examination saving the patient both monetary expense as well asthe discomfort and increased hazard of a physical probe. Other examplesof organs that can be examined include the lungs, stomach and portionsof the gastrointestinal system, the heart and blood vessels.

As shown in FIG. 1, a method for performing a virtual examination of anobject such as a colon is indicated generally by the reference numeral100. The method 100 illustrates the steps necessary to perform a virtualcolonoscopy using volume visualization techniques. Step 101 prepares thecolon to be scanned in order to be viewed for examination if required byeither the doctor or the particular scanning instrument. Thispreparation could include cleansing the colon with a “cocktail” orliquid, which enters the colon after being orally ingested and passedthrough the stomach. The cocktail forces the patient to expel wastematerial that is present in the colon. One example of a substance usedis Golytcly. Additionally, in the case of the colon, air or carbondioxide can be forced into the colon in order to expand it to make thecolon easier to scan and examine. This is accomplished with a small tubeplaced in the rectum with approximately 1,000 cc of air pumped into thecolon to distend the colon. Depending upon the type of scanner used, itmay be necessary for the patient to drink a contrast substance such asbarium to coat any unexpunged stool in order to distinguish the waste inthe colon from the colon walls themselves. Alternatively, the method forvirtually examining the colon can remove the virtual waste prior to orduring the virtual examination as explained later in this specification.Step 101 does not need to be performed in all examinations as indicatedby the dashed line in FIG. 1.

Step 103 scans the organ that is to be examined. The scanner can be anapparatus well known in the art, such as a spiral CT-scanner forscanning a colon or a Zenith MRI machine for scanning a lung labeledwith xenon gas, for example. The scanner must be able to take multipleimages from different positions around the body during suspendedrespiration, in order to produce the data necessary for the volumeVisualization. For example, data can be acquired using a GE/CTI spiralmode scanner operating in a helical mode of 5 mm, 1.5-2.0:1 pitch,reconstructed in 1 mm slices, where the pitch is adjusted based upon thepatient's height in a known manner. A routine imaging protocol of 120kVp and 200-280 ma can be utilized for this operation. The data can beacquired and reconstructed as 1 mm thick slice images having an arraysize of 512×512 pixels in the field of view, which varies from 34 to 40cm depending on the patient's size. The number of such slices generallyvaries under these conditions from 300 to 450, depending on thepatient's height. The image data set is converted to volume elements orvoxels.

An example of a single CT-image would use an X-ray beam of 5 mm width,1:1 to 2:1 pitch, with a 40 cm field-of-view being performed from thetop of the splenic flexure of the colon to the rectum.

Discrete data representations of the object can be produced by othermethods besides scanning. Voxel data representing an object can bederived from a geometric model by techniques described in U.S. Pat. No.5,038,302 entitled “Method of Connecting Continuous Three-DimensionalGeometrical Representations into Discrete Three-Dimensional Voxel-BasedRepresentations Within a Three-Dimensional Voxel-Based System” byKaufman, issued Aug. 8, 1991, filed Jul. 26, 1988, which is herebyincorporated by reference in its entirety. Additionally, data can beproduced by a computer model of an image, which can be converted tothree-dimensional voxels and explored in accordance with thisdisclosure.

Step 104 converts the scanned images into three-dimensional volumeelements (“voxels”). In a preferred embodiment for examining a colon,the scan data is reformatted into 5 mm thick slices at increments of 1mm or 2.5 mm and reconstructed in 1 mm slices, with each slicerepresented as a matrix of 512 by 512 pixels. By doing this, voxels ofapproximately 1 cubic mm are created. Thus a large number of 2D slicesare generated depending upon the length of the scan. The set of 2Dslices is then reconstructed to 3D voxels. The conversion process of 2Dimages from the scanner into 3D voxels can either be performed by thescanning machine itself or by a separate machine such as a computerimplementing techniques that are well known in the art (see, e.g., U.S.Pat. No. 4,985,856 entitled “Method and Apparatus for Storing,Accessing, and Processing Voxel-based Data” by Kaufman et al.; issuedJan. 15, 1991, filed Nov. 11, 1988; which is hereby incorporated byreference in its entirety).

Step 105 allows the operator to define the portion of the selected organto be examined. A physician may be interested in a particular section ofthe colon likely to develop polyps. The physician can view a twodimensional slice overview map to indicate the section to be examined. Astarting point and finishing point of a path to be viewed can beindicated by the physician/operator. A conventional computer andcomputer interface (e.g., keyboard, mouse or spaceball) can be used todesignate the portion of the colon that is to be inspected. A gridsystem with coordinates can be used for keyboard entry or thephysician/operator can “click” on the desired points. The entire imageof the colon can also be viewed if desired.

Step 107 performs the planned or guided navigation operation of thevirtual organ being examined. Performing a guided navigation operationis defined as navigating through an environment along a predefined orautomatically predetermined flight path, which can be manually adjustedby an operator at any time. After the scan data has been converted to 3Dvoxels, the inside of the organ is traversed from the selected start tothe selected finishing point. The virtual examination is modeled onhaving a tiny viewpoint or “camera” traveling through the virtual spacewith a view direction or “lens” pointing towards the finishing point.The guided navigation technique provides a level of interaction with thecamera, so that the camera can navigate through a virtual environmentautomatically in the case of no operator interaction, and at the sametime, allow the operator to manipulate the camera when necessary. Thepreferred embodiment of achieving guided navigation is to use aphysically based camera model that employs potential fields to controlthe movement of the camera, as is further detailed with respect to FIG.2.

Step 109, which can be performed concurrently with step 107, displaysthe inside of the organ from the viewpoint of the camera model along theselected pathway of the guided navigation operation. Three-dimensionaldisplays can be generated using techniques well known in the art such asthe marching cubes technique, for example. In order to produce a realtime display of the colon, a technique is used that reduces the vastnumber of data computations necessary for the display of the virtualorgan.

The method described in FIG. 1 can also be applied to scanning multipleorgans in a body at the same time. For example, a patient may beexamined for cancerous growths in both the colon and lungs. The methodof FIG. 1 would be modified to scan all the areas of interest in step103 and to select the current organ to be examined in step 105. Forexample the physician/operator may initially select the colon tovirtually explore and later explore the lung. Alternatively, twodifferent doctors with different specialties may virtually exploredifferent scanned organs relating to their respective specialties.Following step 109, the next organ to be examined is selected and itsportion will be defined and explored. This continues until all organsthat need examination have been processed.

The steps described in conjunction with FIG. 1 can also be applied tothe exploration of any object that can be represented by volumeelements. For example, an architectural structure or inanimate objectcan be represented and explored in the same manner.

Turning to FIG. 2, a “submarine camera” model that performs guidednavigation in a virtual organ is indicated generally by the referencenumeral 200. The model 200 depicts a viewpoint control model thatperforms the guided navigation technique of step 107. When there is nooperator control during guided navigation, the default navigation issimilar to that of planned navigation that automatically directs thecamera along a flight path from one selected end of the colon toanother. During the planned navigation phase, the camera stays at thecenter of the colon for obtaining better views of the colonic surface.When an interesting region is encountered, the operator of the virtualcamera using guided navigation can interactively bring the camera closeto a specific region and direct the motion and angle of the camera tostudy the interesting area in detail, without unwillingly colliding withthe walls of the colon. The operator can control the camera with astandard interface device such as a keyboard, mouse or non-standarddevice such as a spaceball. In order to fully operate a camera in avirtual environment, six degrees of freedom for the camera are required.The camera must be able to move in the horizontal, vertical, and depthor Z direction (axes 217), as well as being able to rotate in anotherthree degrees of freedom (axes 219) to allow the camera to move and scanall sides and angles of a virtual environment.

Methods for computing a centerline inside the area of interest are wellknown in the art (see, e.g., U.S. Pat. No. 5,971,767 entitled “SYSTEMAND METHOD FOR PERFORMING A THREE-DIMENSIONAL VIRTUAL EXAMINATION” byKaufman et al.; issued Oct. 26, 1999 and incorporated by referenceherein in its entirety).

Referring to FIG. 3, a two dimensional cross-section of a volumetriccolon containing a flight path is indicated generally by the referencenumeral 300. The cross-section 300 includes the final flight path forthe camera model down the center of the colon, as indicated by “x”s, andat least one starting location 301 or 303 near one end of the colon.

Turning now to FIG. 4, a system used to perform a virtual examination ofa human organ in accordance with the disclosure is indicated generallyby the reference numeral 400. The system 400 is for performing thevirtual examination of an object such as a human organ using thetechniques described herein. A patient 401 lays on a platform 402, whilea scanning device 405 scans the area that contains the organ or organsto be examined. The scanning device 405 contains a scanning portion 403that takes images of the patient and an electronics portion 406. Theelectronics portion 406 includes an interface 407, a central processingunit 409, a memory 411 for temporarily storing the scanning data, and asecond interface 413 for sending data to a virtual navigation platformor terminal 416. The interfaces 407 and 413 may be included in a singleinterface component or may be the same component. The components in theportion 406 are connected together with conventional connectors.

In the system 400, the data provided from the scanning portion 403 ofthe device 405 is transferred to unit 409 for processing and is storedin memory 411. The central processing unit 409 converts the scanned 2Ddata to 3D voxel data and stores the results in another portion of thememory 411. Alternatively, the converted data may be directly sent tothe interface unit 413 to be transferred to the virtual navigationterminal 416. The conversion of the 2D data could also take place at thevirtual navigation terminal 416 after being transmitted from theinterface 413. In the preferred embodiment, the converted data istransmitted over a carrier 414 to the virtual navigation terminal 416 inorder for an operator to perform the virtual examination. The data mayalso be transported in other conventional ways, such as storing the dataon a storage medium and physically transporting it to terminal 416 or byusing satellite transmissions, for example.

The scanned data need not be converted to its 3D representation untilthe visualization-rendering engine requires it to be in 3D form. Thissaves computational steps and memory storage space.

The virtual navigation terminal 416 includes a screen for viewing thevirtual organ or other scanned image, an electronics portion 415 and aninterface control 419 such as a keyboard, mouse or spaceball. Theelectronics portion 415 includes an interface port 421, a centralprocessing unit 423, optional components 427 for running the terminaland a memory 425. The components in the terminal 416 are connectedtogether with conventional connectors. The converted voxel data isreceived in the interface port 421 and stored in the memory 425. Thecentral processing unit 423 then assembles the 3D voxels into a virtualrepresentation and runs the submarine camera model as described for FIG.2 to perform the virtual examination. As the submarine camera travelsthrough the virtual organ, the visibility technique is used to computeonly those areas that are visible from the virtual camera, and displaythem on the screen 417. A graphics accelerator can also be used ingenerating the representations. The operator can use the interfacedevice 419 to indicate which portion of the scanned body is desired tobe explored. The interface device 419 can further be used to control andmove the submarine camera as desired as detailed for FIG. 2. Theterminal portion 415 can be, for example, the Cube-4 dedicated systembox, generally available from the Department of Computer Science at theState University of New York at Stony Brook.

The scanning device 405 and terminal 416, or parts thereof, can be partof the same unit. A single platform would be used to receive the scanimage data, connect it to 3D voxels if necessary and perform the guidednavigation.

An important feature in system 400 is that the virtual organ can beexamined at a later time without the presence of the patient.Additionally, the virtual examination could take place while the patientis being scanned. The scan data can also be sent to multiple terminals,which would allow more than one doctor to view the inside of the organsimultaneously. Thus a doctor in New York could be looking at the sameportion of a patient's organ at the same time with a doctor inCalifornia while discussing the case. Alternatively, the data can beviewed at different times. Two or more doctors could perform their ownexamination of the same data in a difficult case. Multiple virtualnavigation terminals could be used to view the same scan data. Byreproducing the organ as a virtual organ with a discrete set of data,there are a multitude of benefits in areas such as accuracy, cost andpossible data manipulations.

Some of the applicable techniques may be further enhanced in virtualcolonoscopy applications through the use of a number of additionaltechniques that are described in U.S. Pat. No. 6,343,936 entitled“SYSTEM AND METHOD FOR PERFORMING A THREE-DIMENSIONAL VIRTUALEXAMINATION, NAVIGATION AND VISUALIZATION” by Kaufman et al.; issuedFeb. 7, 2002, which is incorporated herein by reference in its entirety.These improvements, described briefly below, include improved coloncleansing, volume rendering, additional fly-path determinationtechniques, and alternative hardware embodiments.

An improved electronic colon cleansing technique employs modified bowelpreparation operations followed by image segmentation operations, suchthat fluid and stool remaining in the colon during a computedtomographic (“CT”) or magnetic resonance imaging (“MRI”) scan can bedetected and removed from the virtual colonoscopy images. Through theuse of such techniques, conventional physical washing of the colon, andits associated inconvenience and discomfort, is minimized or completelyavoided.

In addition to image segmentation and texture mapping, volume-renderingtechniques may be used in connection with virtual colonoscopy proceduresto further enhance the fidelity of the resulting image. Methods forvolume rendering are well known to those of ordinary skill in thepertinent art.

Turning to FIG. 5, a system embodiment based on a personal computer busarchitecture is indicated generally by the reference numeral 500. Thesystem 500 includes an alternate hardware embodiment suitable fordeployment on a personal computer (“PC”), as illustrated. The system 500includes a processor 510 that preferably takes the form of a high speed,multitasking processor, such as, for example, a Pentium III processoroperating at a clock speed in excess of 400 MHZ. The processor 510 iscoupled to a conventional bus structure 520 that provides for high-speedparallel data transfer. Also coupled to the bus structure 520 are a mainmemory 530, a graphics board 540, and a volume rendering board 550. Thegraphics board 540 is preferably one that can perform texture mapping,such as, for example, a Diamond Viper v770 Ultra board manufactured byDiamond Multimedia Systems. The volume rendering board 550 can take theform of the VolumePro board from Mitsubishi Electric, for example, whichis based on U.S. Pat. Nos. 5,760,781 and 5,847,711, which are herebyincorporated by reference in their entirety. A display device 545, suchas a conventional SVGA or RGB monitor, is operably coupled to thegraphics board 540 for displaying the image data. A scanner interfaceboard 560 is also provided for receiving data from an imaging scanner,such as an MRI or CT scanner, for example, and transmitting such data tothe bus structure 520. The scanner interface board 560 may be anapplication specific interface product for a selected imaging scanner orcan take the form of a general-purpose input/output card. The PC basedsystem 500 will generally include an I/O interface 570 for coupling I/Odevices 580, such as a keyboard, digital pointer or mouse, and the like,to the processor 510. Alternatively, the I/O interface can be coupled tothe processor 510 via the bus 520.

In practical application, radiological scanning equipment is used toacquire respective scan data from a patient from both a supine (i.e.,face up) and prone (i.e., face down) position. Other than being gatheredfrom the same patient, the image data corresponding to the twoorientations are completely independent. As a consequence, when aninteresting feature is located in one orientation or view, there iscurrently no way to automatically “jump” to that same feature in theother orientation or view. To enable meaningful switching between to thetwo views, the underlying image data must be linked, or associated insome way. The colon, for example, is a flexible organ that may changeits geometry significantly when the a patient is re-oriented and, thus,the problem falls into the class known as non-rigid image registrationproblems.

A number of different approaches can be used to register the two viewsof a virtual organ. However, to be useful, the image registrationalgorithm should be computationally efficient. For that reason,embodiments of the present disclosure focus on methods that useinformation regarding the centerline of the organ to establish acorrespondence between the points on each centerline. The centerline ofthe virtual organ can be calculated according to any of the previouslydescribed methods or any other method as known to those of ordinaryskill in the pertinent art. This focused approach effectively reducesthe problem to one-dimension and, thus, is computationally efficient.

A first step in establishing a correspondence between the supine-viewand prone-view centerlines is to identify features that are common toboth. The correspondence, in the different views, of points between suchlandmark features is then performed using simple interpolation (e.g.,linear, higher order, and the like) techniques that can be found incomputer science literature (see, e.g., Brown, L. G. 1992, A Survey ofImage Registration Techniques, ACM Computing Surveys 24(4), 325-376).

There are a few possibilities for elements that constitute a featurealong the centerline. Examples include the hepatic flexure, the splenicflexure, end points of the centerline, internal hostral folds, and bendsand curves of the lumen. Of these possibilities, the hepatic flexure,which generally has the highest “z” value along the center-line, or thesplenic flexure, which generally has a lower “z” value along thecenter-line, provide an easy means for achieving overall, coarseregistration between the two center-lines. In addition, embodiments ofthe present disclosure utilize the bends or changes of direction of acenterline as good features for addressing the registration problem. Asthe colon tends to bend only in a few places, the location of thesebending places may be used to perform registration.

In the following exemplary embodiment, registration between the supineand prone centerlines is described from the perspective of matching thesupine data to that of the prone data. This matching of data can also beperformed in the opposite manner.

An initial step for registering the two centerlines is to locate, oneach centerline, the bending features. The method of computation is thesame for both the prone-view data and the supine-view data. Inparticular, the change in direction of the centerline is computed foreach voxel, v_(i), along the centerline, where v_(i)=(x_(i), y_(i),z_(i)).

Referring to FIG. 6, a plot of voxel coordinates versus centerlineposition is indicated generally by the reference numeral 600, which ishelpful in understanding the computation of the change in direction. Thegraph of FIG. 6 has, as its abscissa, the centerline position of thevoxel or, in other words, the “i” of v_(i). The ordinate axis is one ofthe voxel coordinates, which for exemplary purposes, is considered to be“z”. Point 602 is a voxel v_(i) along the centerline. Using this point602, the vectors a, b and r are computed where:a=v _(i−k) −v _(i)b=v _(i) +v _(i+k)r=b−a

The length of the difference vector, r, for each voxel point defines thechange in direction about the voxel point, such as, for example, thepoint 602. The value “k” is a constant and can typically be around 100for data acquired using convention CT scanning devices.

As shown in FIGS. 7A and 7B, supine and prone centerline plots depictinga “z” profile of the centerline overlaid with a plot of the change indirection of the “z” profile are indicated generally by the referencenumerals 700A and 700B, respectively. Both plots 700A and 700B depictthe above-described algorithm applied to actual CT scan data with k=100.FIG. 7A relates to supine-view data, while FIG. 7B relates to prone-viewdata. The smoother lines 702 a and 702 b reflect, for each respectiveview, the z coordinate of each voxel point along the centerline. Theother lines 704 a and 704 b reflect, for each respective view, themagnitude of the direction of change around each voxel (i.e., the vectorr).

From each of the lines 704 a and 704 b, the locations of significantbends are extracted to serve as features for registering the twocenterlines. One algorithm for identifying those features that aresignificant involves analyzing each point, a_(i), along the line 704 aor 704 b. The analysis involves determining whether the following threeconditions are satisfied:a _(i−j) <=a _(i)a _(i+j) <=a _(i)a _(i)−0.5(a _(i−j) +a _(i+j))>=T

Here, T is an empirically determined threshold value and j is a constantthat may, or may not be, the same as the previously used k. A commonvalue for j is j=1. Any voxel, v_(i), whose change in direction, a_(i),satisfies these equations represents a significant feature along thecenterline. Examples of these significant features are depicted as dots710 a and 710 b in FIGS. 7A and 7B, respectively.

Once the features on each centerline are extracted, or identified, thefeatures can be matched to one another. Once matched, a position in oneview within a certain proximity of a feature can be estimated in theother view based on the matched feature.

Turning to FIG. 8A, a voxel z profile for both a supine view and a proneview is indicated generally by the reference numeral 800. The z profile800 covers both a supine view and a prone view with no manipulation ofthe significant features 802 in either centerline. Because thecenterline data from each view is unrelated and the number ofsignificant features is unlikely to be equal, it is not meaningful tosimply match the first feature of the supine view with that of the proneview and proceed until all features are thus matched. FIG. 8B shows asimple aligning of the two centerlines at their hepatic flexures byshifting, or translating, the supine view until its hepatic flexure isaligned with the hepatic flexure of the prone view. Once this coarsealignment is done, correspondence between features can be performedaccording to the following “simple” matching algorithm:

-   -   (a) For every significant feature f_(n) ^(S) in the supine view,        find the closest, prone-view feature f_(m) ^(P);    -   (b) Determine if there is any other significant feature in the        supine view that is closer to f_(m) ^(P) than f_(n) ^(S); and    -   (c) If there is no closer significant feature in the supine        view, then associate, or match, f_(m) ^(P) with f_(n) ^(S);        otherwise ignore f_(n) ^(S).

When determining whether a feature is closer than another feature, it ismerely in reference to their location, “i” along the abscissa of FIGS.8A and 8B. For example, a feature f₇ ^(P) 804 corresponds to a voxelv_(c) in the prone data set. Similarly, features 806 (e.g., f₇ ^(S)) and808 (e.g., f₈ ^(S)) each correspond to different voxels, v_(a) andv_(b), in the supine data set. Closeness is simply a measure of whetherthe index of the voxel (e.g., “a” or “b”) is closer to “c”.

The results from this coarse alignment and matching can achieve locationof significant features within a few centimeters in many pairs of datasets. According to another embodiment of the present disclosure, thealigning and matching algorithm is more complex in order to improve theaccuracy of matching significant features along different centerlines,especially as these features get farther and farther from the hepaticflexure.

According to this alternate embodiment, the “simple” feature-matchingalgorithm is applied incrementally from the hepatic flexure according tothe algorithm:

-   -   (a) match the locations of the hepatic flexures for each        centerline as described above;    -   (b) while significant features exits in the supine data to the        right of the hepatic flexure:        -   (b)(1) apply the “simple” matching algorithm to the next            supine significant feature. If a match is made, then            translate, or shift, the supine-view data so that the            matched features f_(n) ^(S) and f_(m) ^(P) are aligned. If            no match occurs, ignore the feature;    -   (c) again match the locations of the hepatic flexures for each        centerline by translating one of the data sets (e.g., the        supine-view data); and    -   (d) while significant features exist in the supine data to the        left of the hepatic flexure:        -   (d)(1) apply the “simple” matching algorithm to the next            supine significant feature. If a match is made, then            translate, or shift, the supine-view data so that the            matched features f_(n) ^(S) and f_(m) ^(P) are aligned. If            no match occurs, ignore the feature.

This is similar to the simple matching algorithm but a key difference isthat on each side of the hepatic flexure, the matching features aresuccessively, incrementally aligned. In the earlier embodiment, thelocation of each feature remained the same in the supine data set oncethe supine data set was translated to match the hepatic flexures.However, in the incremental matching algorithm, each match of a featureto the right of the hepatic flexure further translates (either right orleft) those features to the right of the current matched feature. Also,each match of a feature to the left of the hepatic flexure results infurther translation, or shifting, of those features to the left of thecurrent matched feature. As a result, for determining the “closest”feature when performing the matching, the locations of the previouslymatched features are taken into account.

Turning out of sequence to FIG. 10, a registration method according toan embodiment of the present disclosure is indicated generally by thereference numeral 1000. In step 1002, a patient is scanned in both theprone and supine positions to acquire respective data sets that are usedto generate respective three-dimensional representations of an organ,such as a colon. In step 1004, a respective centerline is identified foreach virtual organ.

Next, in step 1006, the significant features in each centerline areidentified. For example, those features that indicate bends or folds ina colon can be considered as significant features. One method fordetermining a bend or fold is to measure the change in direction aroundeach voxel; if this measure exceeds a threshold then that voxel isconsidered to be a significant feature.

Features from one centerline are then matched, in step 1008, to arespective feature in the other centerline. One coarse registrationmethod is to align the respective hepatic flexures and then matchfeatures by finding the feature in one data set that is closest to eachfeature in the other dataset. Another method works outwardly from thehepatic flexure and incrementally shifts one data set's position as afeature in that data set is matched to a feature in the other data set.

In step 1010, an operator can utilize the matched feature information toarrive at the same relative location when jumping from the supine-viewvirtual organ to the prone-view virtual organ.

Sometimes, the colon segmentation algorithm thus described produces twoor more disjoint colon segments. The algorithms described above wouldthus work only for segments that included the hepatic flexure.

Referring now to FIG. 9, a plot of views each having two disjointsegments is indicated generally by the reference numeral 900. The plot900 depicts a supine view (similar to that of FIG. 7A) and a prone view(similar to that of FIG. 7B), each having two disjoint segments. Thesupine view has segments 900 a and 900 b, which have endpoints 902 and904, respectively. The prone view has segments 901 a and 902 b withendpoints 906 and 908, respectively. In order to use the above matchingalgorithms, the two segments of each view are connected by linearinterpolation between the two endpoints. For example, in the supineview, a line segment 910 is added to marry the segments 900 a and 900 b.Similarly, a line segment 912 is added to the prone view. These sameprinciples can easily be applied to more than two disjoint segments. Asa result, the hepatic flexures can be aligned and matching commenced. Ina preferred embodiment, matching is performed only for the originalsegments, not the interpolated regions.

An additional embodiment of the disclosure modifies the constant “k”near the endpoint of a colon segment. The features that are matched bythe previously described registration algorithms are the peaks in thecurve that reflected the change in direction about a point. To calculatethis for each point, these algorithms looked ahead and behind a point by“k” voxels, where was such as about 100 voxels, for example. Thus, thechange-in-direction data begins and ends 100 voxels from a segment startand end, respectively. Furthermore, there is a 200-voxel gap wheredisjoint segments meet. In order to make use of more data near segmentendpoints and to be able to analyze shorter segments, “k” decreaseslinearly towards some minimum value as an endpoint is approached. Forexample, if k_(max)=100 and k_(min)=10, the matching algorithms can beused to compute change in direction values to within 10 voxels of theendpoints and the gap between disjoint segments is thereby reduced to 20voxels.

The value “j” described above when determining those features that aresignificant is used to determine how many voxels ahead of and behind avoxel to look. While j=1 provides the best resolution, it can sometimesresult in a cluster of features that actually represent only a singlefeature. Increasing the value to j=10 still results in a feature at eachpeak within the change-in-direction curve (e.g., 704 a), and falseclusters of features can be avoided.

A similar effect can be achieved using a value representing a minimumdistance between features (“MDBF”). For example if the MDBF=3, anyfeatures without 3 voxels between them are merged into a single cluster.

In some instances, the first feature to the right (or left) of thehepatic flexure may be a relatively large distance away. In these cases,the incremental matching algorithm may not provided the desiredaccuracy. One alternative for addressing these cases is to provide agraphical interface to an operator that displays the supine and proneplots with features shown. The operator can then select two features (asopposed to the hepatic flexures) to use for seeding the initial coarsealignment. Another alternative is to define the hepatic flexure on eachplot as the feature with the highest “z” value, rather than just thevoxel with the highest z value.

Once the respective features of each plot are identified and matched,then the goal of switching between the supine and prone views can beachieved. While navigating a virtual organ, such as a colon, based oneither a supine or a prone data set according to the system and methodsdescribed herein, the operator can click on a position. Using thelocation and distance of the closest matched feature, the equivalent ofthe original position in the alternate view is displayed within avirtual organ c.

The foregoing merely illustrates the principles of the disclosure. Itwill thus be appreciated that those skilled in the art will be able todevise numerous systems, apparatus and methods which, although notexplicitly shown or described herein, embody the principles of thedisclosure and are thus within the spirit and scope of the disclosure asdefined by its Claims.

For example, the methods and systems described herein could be appliedto virtually examine an animal, fish or inanimate object. Besides thestated uses in the medical field, applications of the technique could beused to detect the contents of sealed objects that cannot be opened. Thetechnique could also be used inside an architectural structure such as abuilding or cavern and enable the operator to navigate through thestructure.

These and other features and advantages of the present disclosure may bereadily ascertained by one of ordinary skill in the pertinent art basedon the teachings herein. It is to be understood that the teachings ofthe present disclosure may be implemented in various forms of hardware,software, firmware, special purpose processors, or combinations thereof.

Most preferably, the teachings of the present disclosure are implementedas a combination of hardware and software. Moreover, the software ispreferably implemented as an application program tangibly embodied on aprogram storage unit. The application program may be uploaded to, andexecuted by, a machine comprising any suitable architecture. Preferably,the machine is implemented on a computer platform having hardware suchas one or more central processing units (“CPU”), a random access memory(“RAM”), and input/output (“I/O”) interfaces. The computer platform mayalso include an operating system and microinstruction code. The variousprocesses and functions described herein may be either part of themicroinstruction code or part of the application program, or anycombination thereof, which may be executed by a CPU. In addition,various other peripheral units may be connected to the computer platformsuch as an additional data storage unit and a printing unit.

It is to be further understood that, because some of the constituentsystem components and methods depicted in the accompanying drawings arepreferably implemented in software, the actual connections between thesystem components or the process function blocks may differ dependingupon the manner in which embodiments of the present disclosure areprogrammed. Given the teachings herein, one of ordinary skill in thepertinent art will be able to contemplate these and similarimplementations or configurations of the present invention.

Although the illustrative embodiments have been described herein withreference to the accompanying drawings, it is to be understood that thepresent invention is not limited to those precise embodiments, and thatvarious changes and modifications may be effected therein by one ofordinary skill in the pertinent art without departing from the scope orspirit of the present invention. All such changes and modifications areintended to be included within the scope of the present invention as setforth in the appended Claims.

1. A method for registering a plurality of three-dimensional volume datasets of at least one object, the method comprising: locating a first setof features along an object centerline within a first three-dimensionalvolume data set; locating a second set of features along an objectcenterline within a second three-dimensional volume data set; andmatching at least one feature in one of the first or second data setswith a corresponding feature in the other data set.
 2. A method asdefined in claim 1 wherein the first and second data sets correspond toa prone view and a supine view, respectively.
 3. A method as defined inclaim 1 wherein the object is an organ within a body.
 4. A method asdefined in claim 3 wherein the organ is a colon.
 5. A method as definedin claim 4 wherein each feature in the first and second sets correspondsto a bend or fold in the colon.
 6. A method as defined in claim 3wherein the organ is a lung.
 7. A method as defined in claim 3 whereinthe organ is a heart.
 8. A method as defined in claim 2, furthercomprising: scanning the at least one object with a scanning device andproducing prone-view scan data representative of the object in a proneorientation; creating the prone-view three-dimensional volumerepresentation from said prone-view scan data; scanning the at least oneobject with a scanning device and producing supine-view scan datarepresentative of the object in a supine orientation; and creating thesupine-view three-dimensional volume representation from saidsupine-view scan data.
 9. A method as defined in claim 1 whereinmatching comprises: for each particular feature in the second set,locating a closest feature in the first set; determining if anotherfeature in the second set is closer to the closest feature than theparticular feature; and if the other feature is closer, then ignoringthe particular feature and matching the other feature with the closestfeature, and otherwise matching the particular feature with the closestfeature.
 10. A method as defined in claim 2 wherein the object is acolon and each centerline has a respective hepatic flexure, and matchingcomprises: shifting the prone-view and supine-view centerlines to alignthe respective hepatic flexures; for each particular feature in thesecond set, locating a closest feature in the first set; determining ifanother feature in the second set is closer to the closest feature thanthe particular feature; and if the other feature is closer, thenmatching the other feature with the closest feature, and otherwisematching the particular feature with the closest feature.
 11. A methodas defined in claim 10, further comprising: performing a guidednavigation of one of the prone or supine view representations along adefined path; displaying volume elements in real time at a currentlocation along the defined path, the volume elements responsive to saidpath and any operator's input during the guided navigation; and aligninga location in one view to a location in the other view based on thecurrent location.
 12. A method as defined in claim 11 wherein jumpingcomprises: identifying a direction and distance of a closest feature tothe current location; and locating the target location based upon thefeature corresponding to the closest feature.
 13. A method forperforming a three-dimensional internal examination of at least oneorgan, comprising: scanning the organ with a radiological device andproducing first and second scan data sets representative of the organ inprone and supine views, respectively; creating a prone-viewthree-dimensional volume representation of the organ comprising volumeelements from the prone-view scan data; creating a supine-viewthree-dimensional volume representation of the organ comprising volumeelements from the supine-view scan data; generating a prone-view organcenterline and a supine-view organ centerline within the respectivethree-dimensional representations; and displaying in real time thevolume elements responsive to the respective centerline and anyoperator's input during a guided navigation along one of the supine-viewor prone-view three-dimensional representations.
 14. A method as definedin claim 13 wherein the organ is a colon.
 15. A method as defined inclaim 13 wherein the organ is a lung.
 16. A method as defined in claim13 wherein the organ is a heart.
 17. A method as defined in claim 13,further comprising: identifying a first set of features along theprone-view centerline; identifying a second set of features along thesupine-view centerline; and matching each feature in one of the first orsecond sets with a corresponding feature in the other set.
 18. A methodas defined in claim 17 wherein each step of identifying furthercomprises: determining for each particular voxel along the respectivecenterline a change-in-direction value; and identifying the particularvoxel as a feature for inclusion in the respective set if itschange-in-direction value satisfies a particular threshold.
 19. A methodas defined in claim 18 wherein determining a change-in-direction valuecomprises: calculating a first vector from a fist voxel, said firstvoxel located k units before the particular voxel along the respectivecenterline wherein k is an integer grater than zero; calculating asecond vector from the particular voxel to a second voxel located kunits after the particular voxel on the respective centerline; andcalculating a change-in-direction value, a₁, for the particular vectorby subtracting the first vector from the second vector.
 20. A method asdefined in claim 19 wherein identifying comprises: determining if thechange-in-direction value, a₁, for the particular voxel is greater thanor equal to a change-in-direction value, a₂, for a third voxel located junits before the particular voxel along the respective centerlinewherein j is an integer greater than zero; determining if thechange-in-direction value, a₁, for the particular voxel is greater thanor equal to a change-in-direction value, a₃, for a fourth voxel locatedj units after the particular voxel along the respective centerline;determining if a quantity responsive to a₁, a₂ and a₃ exceeds or equalsthe predetermined threshold; and identifying the particular voxel as afeature within the respective set if all three determinations aresatisfied.
 21. A method as defined in claim 20 wherein the quantity is[a₁−0.5(a₂+a₃)].
 22. A method as defined in claim 17 wherein the organis a colon and each centerline has a respective hepatic flexure, andmatching comprises: shifting the prone-view and supine-view centerlinesto align the respective hepatic flexures; for each particular feature inthe other set, locating a closest feature in the one set; determining ifanother feature in the other set is closer to the closest feature thanthe particular feature; and if the another feature is closer, thenmatching the another feature with the closest feature, and otherwisematching the particular feature with the closest feature.
 23. A methodas defined in claim 17 wherein the organ is a colon and each centerlinehas a respective hepatic flexure, and matching comprises: (a) shiftingthe prone-view and supine-view centerlines to align the respectivehepatic flexures; (b) repeating the following steps for each featurewithin the first set located on a first side of the hepatic flexure: (1)for a current feature in the first set, find a closest feature in thesecond set wherein the current feature is the feature closest, on thefirst side, to the hepatic flexure; (2) determine if another feature inthe first set is closer to the closest feature than the current feature;(3) if another feature is closer, then matching the another feature withthe closest feature, and otherwise matching the current feature with theclosest feature; (4) translating a position of the features in thesecond set to align the matched features; (5) making the next closestfeature on the first side the current feature; (c) repeating thefollowing steps for each feature within the first set located on asecond side of the hepatic flexure: (1) for a current feature in thefirst set, find a closest feature in the second set wherein the currentfeature is the feature closest, on the second side, to the hepaticflexure; (2) determine if another feature in the first set is closer tothe closest feature than the current feature; (3) if another feature iscloser, then matching the another feature with the closest feature, andotherwise matching the current feature with the closest feature; (4)translating a position of the features in the second set to align thematched features; and (5) making the next closest feature on the secondside the current feature.
 24. A method as defined in claim 17, furthercomprising: jumping from a current location in the displayed onethree-dimensional representation to a target location in the otherthree-dimensional representation; and displaying the otherthree-dimensional representation.
 25. A method as defined in claim 24wherein jumping further comprises: identifying a current feature in therespective set closest to the current location; calculating a distanceand direction of the current feature with respect to the currentlocation; and calculating the target location based on the distance, thedirection and a target feature corresponding to the current feature. 26.A program storage device readable by machine, tangibly embodying aprogram of instructions executable by the machine to perform methodsteps for registering a plurality of three-dimensional volume data setsof at least one object, the method steps comprising: locating a firstset of features along a centerline within a first three-dimensionalvolume data set; locating a second set of features along a centerlinewithin a second three-dimensional volume data set; and matching at leastone feature in one of the first or second data sets with a correspondingfeature in the other data set.
 27. An apparatus for registering aplurality of three-dimensional volume data sets of at least one object,the apparatus comprising: first locating means for locating a first setof features along a centerline within a first three-dimensional volumedata set; second locating means for locating a second set of featuresalong a centerline within a second three-dimensional volume data set;and matching means for matching at least one feature in one of the firstor second data sets with a corresponding feature in the other data set.28. An apparatus for registering a plurality of three-dimensional volumedata sets of at least one object, the apparatus comprising: a scanningdevice for receiving a plurality of three-dimensional volume data setsof at least one object; a processing device in signal communication withthe scanning device for locating a first set of features along acenterline within a first three-dimensional volume data set and locatinga second set of features along a centerline within a secondthree-dimensional volume data set; and a computing device in signalcommunication with the processing device for matching at least onefeature in one of the first or second data sets with a correspondingfeature in the other data set.